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"RIDGE  ANALYSIS"  OF  RESPONSE  SURFACES* 

Norman  R.  Draper 
University  of  Wisconsin 

0.  Introduction.  In  a  1959  paper,  A.  E.  Hoerl  discussed  a 
method  for  examining  a  second  order  response  surface.  This  paper 
provides  a  mathematically  simpler  derivation  of  the  technique 
and  proofs  of  some  stated  properties. 

1.  Lagrange1 a  Undetermined  Multipliers. 

A  well-known  (e.g.  Kaplan,  1956)  method  of  obtaining  the 
stationary  or  turning  values  of  a  function  f  (xi,x2, . .  .,x^)  of  k 
variables  xi,x2, ... .,  x^  subject  to  restrictions  on  the  x.^  such 

gj (xi,x2, . . .  xk)  =0,  (j  *  1,2, ...n) 

is  the  following.  Form  the  function 

n 

F  -  f  -  2  X.g,  (1.1) 

j-1  3  3 

where  Xi,  X2,  ...  Xfl  are  arbitrary.  Differentiate  (1.1) 
partially  with  respect  to  each  x^  and  set  the  results  equal  to 
zero.  This  will  provide  the  k  equations 

This  research  was  supported  by  the  United  States  Navy  through 
the  office  of  Naval  Research,  under  Contract  Nonr  -  1202(17), 
Project  NR  042  222. 
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SF _  _  df  °  ,  dg.  _  n 

dx ,  ~  §x .  1  j  ^  ~  ■* 

i  i  j=l  J  dxi 


(1.2) 


(i  =  1,2,...,  k) 

These  k  equations,  with  the  additional  n  equations 

gj  »  0  ,  (.1  =  1,  2,  .  .  .,  n)  (1.3) 

provide  (n  +  k)  equations  which  can  be  solved  for  the  (n  +  k) 


unknowns  Xj.,x2,  .  .  .  ,x  ,  Xj.,X2, .  . .  ,X  .  Often  the  quantities 

iC  n 

X  are  eliminated  and  not  actually  found;  for  this  reason  the 

J 

words  unde  ter  mined  multipliers’’  are  used  to  describe  them. 

In  some  cases,  however,  the  solutions  for  xi,x2,...,x,  are  easier 
to  obtain  if  the  X^  are  evaluated  firot;  in  other  ca„es,  as 
below,  it  may  be  easier  to  specify  values  of  X,  in  equations 
(1.2)  and  regard  other  quantities  in  equations  (1.3)  as  : 'unde¬ 
termined1',  in  their  place. 

Suppose,  now  that  (xi,x2, . )  .  ,x^)  =  (aj.,a2,  •  .  •  ,a^)  is  a 
solution  of  equations  (1.2)  and*  (1.3)  after  elimination  of  X^ . 


Let  M(x)  a  M(xi,x2,...x,  )  = 


82F 

d2F 

82F 

Sxi2 

&Xi&X2 

8xxdx^ 

d2F 

82F 

82F 

dx23xi 

8x28x^ 

•  *  •  • 

82F 

dzF 

8zF 

(1.4) 


j  dx^dx  i  dx^dx2‘  ‘ "  ’  cbc^ 
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be  the  tnatrix  of  Second  order  partial  derivatives.  Then  if 
M(ai,a2, . . . &^)  =  M(a),  the  resulting  matrix  after  the  solution 

a'  ">  (»i,aa, . * .a^)  has  been  substituted  into  (1.4)  is 

(a)  positive  definite,  ie  M  ^  >  0, 

(b)  negative  definite,  ie  y1  M  y  <  0 

where  jr1  =*  (yi,ys,--‘,  y^)  ia  any  1  by  k  real  vector, 
the  function  f (xi,x2, . . . ,x^)  achieves 

(a)  a  local  minimum 

(b)  a  local  maximum. 

respectively.  For,  if  we  expand  F  about  a  as  a  Taylor  series 
of  partial  derivatives,  remembering:  that  all  first  partial  de¬ 
rivatives  of  F  are  zero  at  x  =  a,  we  see  that 
F(a+h)  -  F (a)  =  £  h'  M(a)h  +  0(h3) 
where  h  represents  a  vector  of  small  increments  all  of  the 
same  order  and  0(h3)  represents  a  remainder  of  third  order  in 
Such  increments.  Thus,  to  order  h2,  if  M(a)  is  positive  definite, 

F(a  +  h)  >  F^a)  ,  for  all  small  h. 

If  h  varies  only  in  such  a  way  that  the  restrictions  are  still 
satisfied,  this  implies  that 

f(a  +  h)  >  f  (a) 

ie,  f(a)  is,  locally,  a  minimum,  subject  to  the  restrictions 
holding.  As  we  can  see  from  this  discussion,  it  might  happen 
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that 


+  h)  £  F(a)  ,  for  all  small  h 

but  £(a  +  h)  >  f(a)  ,  for  all  h  which  satisfy  the 

restrictions.  Thus  "M(a)  is  positive  definite"  is  Sufficient, 
but  not  necessary  for  a  local  restricted  minimum  of  f  at  x  *  a. 
Similar  remarks  apply  to  the  negative  definite  case.  If  M(a)  is 
indefinite,  further  investigation  of  the  function  near  the  point 
a  Is  required  to  determine  what  sort  of  stationary  point  has 
been  obtained. 


2.  Improved  derivation  of  the  technique. 

Consider  the  second  order  response  surface  in  k  variable# 
Xi,x2, . . . ,x  ,  given  by 

A, 

y  -  bQ  +  bxxi  +  b2x2  +  .  .  .  +  bkxk 

+  buXi2  +  b22x22  +  ...  +  ^kkxk2  (2*1) 

+  bi2xix2  +  ...  +  bfc  xk.1xk* 

The  point  (0,0,..., 0)  is  the  origin  of  measurement  of  the 
variables  xi,x2,...x(  .  If  the  data  used  to  obtain  (2.1)  re- 

Xv 

suited  from  a  designed  experiment,  it  would  usually  be  the  center 
of  the  design  also.  Suppose  now  we  imagine  a  sphere,  center  at 
the  origin  (0,0,..., 0)  and  of  radius  R,  drawn  in  the  x-space. 
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Then  at  some  points  on  the  sphere  there  will  be  a  maximum  y  and 
elsewhere  a  minimum  "y,  and  possibly  also  (depending  on  the  type 
of  quadratic  surface  (2.1)  obtained  and  the  value  of  R)  values 
of  9  which  are  local  maxima  or  minima,  that  is,  maxima  or  minima 
for  all  nearby  points  on  the  sphere,  but  not  absolute  maxima  and 
minima  when  all  points  of  the  sphere  are  taken  into  consideration. 
If  we  investigate  the  stationary  values  of  the  function  y  on  the 
sphere,  ie  the  stationary  values  subject  to  the  restriction 

g(xi,x2,  • .  • ,  x.R)  s  Xi2  +  x22  +  .  .  .  +  x^2  -  R2  =  0,  (2.2) 

we  shall  be  able  to  find  all  these  local  and  absolute  maxima 
and  minima. 

We  can  then  plot  against  R  as  abscissa  the  following  (k+l) 
ordinates:  Xi,x2, ...,  x^,  y  for,  say,  the  absolute  maximum  of  y 
found  on  the  sphere  radius  R. 

If  we  change  R  slightly  the  appropriate  values  of  Xi,x2,..., 
x^  and  y  for  the  absolute  maximum  will  also  change  slightly  and  so, 
by  varying  R,  we  can  construct  (k+l)  curves  showing  how  the 
position  and  magnitude  of  the  absolute  maximum  y  change  as  R 
changes.  We  can  thus  find,  for  any  selected  R,  the  place  of 
maximum  yield  on  the  response  surface.  Such  a  plot  can  also  be 
made  of  absolute  minimum or  of  the  loci  of  intermediate  stationary 
valuss,  as  desired.  Mathematically,  then,  we  wish  to  find  the 
stationary  valuas  of  y  =  f (xltxSt . . .x^) ,  from  equation  (2.1), 
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subject  to  tha  restriction  g(xi,x2, . . .  ,x^)  =  0  as  in  equation 

(2.2). 

Uging  the  method  of  Lagrange  multipliers  we  set  F  =  y  -  Xg 
and  equations  (1.2)  after  rearrangement  and  division  by  a  factor 
of  2  becpme 


(bn  -  X)  xi  +  ^b12x2  +  .  •  .. 

•+  ^kxk  =  " 

• 

■Jb 1 2x i  +  (b22  -  X)x2  +  ... 

+  -^2kxk  =  " 

■^>2 

(2.3) 

■£b  lkxi  +  -£b  2k  +  . .  . 

+  (bkk-xK 

=  ^bk 

or  in  matrix  notation 

(B  -  X  I)x  =  -  (2.4) 

where 


B  = 

bn 

■^bi2 

....  "^bik 

b  = 

"bx 

■§b12 

b  2  2 

•  •  •  •  2jj 

y 

b2 

■^bj-k  ^b2k 

o' 

7? 

7? 

1 

> 

and  1^  Is  the  k  by  k  unit  matrix. 


#  ’ 

Note:  If  we  set  k=3  and  a  =  2(X  -  b33),  ie  X  =  +  b33, 

we  reduce  to  the  unsynsnetrical  equations  obtained  by  Hoerl 
with  a  as  parameter. 
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Then,  theoretically,  the  (k+l)  equations  (2.4)  and  (2.2)  can  be 
solved  for  sets  of  x.i,x2, . .  .x^,  and  X  corresponding  to  the  vari¬ 
ous  stationary  values  of  y  .#h  the  sphere  radius  R.  Since  the 
solution  in  this  form  leads  to  involved  calculations,  a  simpler 
and  equivalent  method  of  solution  may  be  used  .as  follows . 

(1)  Regard  R  as  variable,  but  fix  X  instead. 

(2)  Insert  the  selected  value  of  X  in  equations  (2.4) 
and  solve  them  for  xi,x2,...x.  .  The  solution  is  used 
In  steps  Sand  4. 

(J)  Compute  R  =  (xia+x22+. . .+  x^2)"^  =  (x1  x)"^,  where  x'  = 
(x!,xa, • • • ,xfc) 

(4)  Evaluate  y. 

We  now  have  a  get  of  numbers  (X,  xi,  x2,  ...,  x  ,  R,  y) 

K. 

and  know  that  on  the  sphere  radius  R,  center  the  origin  there 
ig  a  stationary  value  of  y,  value  determined,  at  the  point 
(xj.,  x2,  ...,  x-^) .  Several  different  values  of  X  will  give  rise 
to  several  stationary  points  which  lie  on  the  same  sphere  radius 
R.  Whether  a  particular  stationary  value  is  the  absolute  maximum, 
absolute  minimum,  a  local  maximum  or  a  local  minimum  is  deter¬ 
mined,  as  we  shall  see,  by  the  value  of  X. 

3.  Properties  of  the  stationary  values. 

Let  the  eigen  values  or  latent  roots  of  the  matrix  B  be 
denoted  by  (i  =  1,  2,...  k) .  Then  the  are  such  that 
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B  x  =  p,  x. 

(3.1) 

or 

(B  -  M-  I)  X  =  £• 

(3-2) 

Hence 

det  (B  -  p.  _l)  =  0, 

(3-3) 

where 

Mdet"  denotes  "the  determinant  of". 

provides  a  kth  degree 

equation  with  root;;  p2,  ...  p^,  *ay.  Note  that  when  a 
standard  canonical  reduction  is  made  of  equation  (2.1), 

.  .  . ,  p^  are  the  latent  roots  needed  to  reduce  y  to  the  form 


y  —  y0  +  piXi2  +  1S-2X2  + 


+  ii  x. 


*k~k  • 

Canonical  reduction  is  another  way  of  examining  a  second  order 
response  surface  for  its  main  features  (C.  L.  Davies,  1956). 

By  comparing  the  value  X,  which  corresponds  to  any  particular 

/\ 

Stationary  value  of  y  on  a  sphere  of  radius  R,  with  the  latent 
roots  M-|.  we  shall  be  able  to  determine  what  sort  of  stationary 
value  hay  been  obtained. 

Suppose  X  =  X 1  and  X  *  X2  are  substituted  in  equation  (2.4) 
and  the  solutions  xi'  =  (ai,a2,  ...,  a^)  and  x2  =  (ci,c2, . . . c^) 
result,  thus  providing  two  stationary  values  yi  and  y2  of  y  on 
the  spheres  x'x  =  R?  and  x'x  =  Rf  ,  respectively.  Then  the 
following  results  are  true. 

a  A 

Result  3-1:  If  Ri  =  R2  and  Xi  >  X2,  the  yx  >ys> 


Proof : 


We  know  that 

(B  -  Xx  I)  Xi  i  -  , 

(B  -  X2  I_)  x2  =  -  -£b  , 


(3.1.1) 

(3.1.2) 
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xi*xi  =  x2  x2  =  R2  ,  say. 

d-i-j) 

yi  =  xi'  B  Xi  +  Xi*  b  +  bQ  , 

(3*1*^) 

and 

ya  -  Xa*  B  x2  +  Xa'  b  +  bQ. 

(3*1*5) 

Premultiplying 

(3.1.1)  and  (3.1.2)  by  xi*  and  x2* 

respectively 

and  subtracting,  and  remembering  (3*1-3) ».  give# 

xx*'.B  xi  -  x2'B  x2  +  *(xi  -  2^2 )  *  b  =  (Xi  -  X2)  R2  ,  (3 •  1*6) 

whence,  using  (3*1*^)  and  (3*1*5)> 

yi  -  y2  =  £(xi  X2) 1  b  +  (Xi  -  X2)  R2  (3*1*7) 

Premultiplying  (3*1*1)  and  (3*1*2)  by  x2'  and.Xi*  respectively 
and  subtracting  gives 

(X2  -  Xi)  Xl'x2  =  i(xi  -  is)'  k  .  (3*  1*8) 

since  x2'  B  xi*  Xi'  B  x2  and  x2'x.i  =  xi'x2  .  Hence  from 
(3.I.7)  and  (3.I.8)  ..  '  ' 

yi  ya  =  (Xi  -  x2)  (R2  -  xs'ii)  (3*1*9) 

But  R2  -  X2*xi  =  (ai2  +  a22  +  ...  +  a^2)"^  (ci2  +  c22  +. . .+  c^  ) 

-  (axci  +  a2c2  +  ...  +  a;<ck)  >  0,  always,  by  a  well-known 

inequality  (Hardy,  Littlewood  and  Polya,  1952) * 

Hence  Xi  >  X2  implies  yi  >  ya* 

Result  3.2:.  If  Ri  =  R2,  M(xj.)  is  positive  definite  and  M(x2) 
ig  indefinite,  then  yi  <  y2* 

Proof :  By  hypothesis 

(B  -  X2  I)  x  £  0  ,  for  at  least  one  2  m  q,  *ay 
y*  (B  -  Xi  I)  y  >  0  ,  for  all  2>  including  2  * 
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Hence  X2  £*£  ££*&£>  Xx 

which  implies  X2  >  Xx. 

By  Result  3.1  then,  yx ' <  y2. 

Similarly,  if  Rx  »  R2,  M(xx)  is:  negative  definite  and  M(x2)  is 
indefinite,  then  yx  >  y2. 

Result  3 •  • If  Xx>  pu  (all  i) ,  then  xx  is  a  point  at  which 
y  attains  a  local  maximum,  on  the  sphere  radius  Rx  ;• if’  Xj.  <  p.^ 

(all  i),  then  Xi  is  a  point  at  which  y  attains  a  local  minimum 
on  the  sphere  radius  Rx.  (As  will  be  seen  later,  we  obtain  the 
absolute  maximum  and  minimum  in  this  way,  not  only  the  local 
maximum  and  minimum. ) 

Proof :  It  will  be  seen  that  equation  (1.4) . becomes 
.  M (x )  «  B  -  Xf  I  ' 

for  the  stationary  point  xx .  Then  if  ^  is  any  n  by  1  vector,  the 
quadratic  form 

i'  M(xx)  £  =  ^'(B  -  Xx  I)  L 

=  -  Xx 

=  2'X  (h  -  xi) 

if  p.  is  any  latent  root  of  B. 

Thus,  if  Xx  >  p.^  ,  (all  i),  M(xx)  is  negative  definite  and 
hence  xx  is  a  point  on  the  sphere  radius  R  at  which  y  attains 
a  maximum,  if  Xx  <  p.^  all  i,  M_(xj. )  is  positive  definite  and 

hence  xx  is  a  point  on  the  sphere,  radius  R  at  which  y  attains  a 


minimum . 


Result  3*4:  Suppose,  as  R  increases,  we  trace  a  locus  of 
stationary  points  (the  absolute  maximum,  absolute  minimum,  or 
a  local  maximum  or  minimum)  and  examine  the  changing 
values  of  y.  Then,  as  R  increases,  $  changes  in  one  of 
the  following  wayo  (when  the  response,  surface' is  quadratic) :  • 

■  ..  (a)  •  decreases  monotonically  •  .  ./  ...  .  • 

;(b)  increases’ monotonically  .  •.  \ 

(c)  passes  through  a  maximum  and  then  decreases  monotonically 

(d)  pauses  through  a  minimum  and  then  increases  monotonically 

If  (c)  and  (d)  happen,  it  is  because  the  locus  has  passed  through 
the  center  of  the  - quadratic  system.  '  ^ 

Proof:  .  y  *  bQ.+.  x*B  x  +  x*  ,b 

■  =  bc  +  X  x'x  t  /•••'  .  :  (2*4.1). 

using  equation  (2.4).  ;  ■  . 

Suppose  we  make  a  small  change  <5X  in  X;  this  will  induce  small 

changes  6x  in  x,  in  aquations  (2.4),  a  small  change  6R  in  R  and 

finally  a  small  change  6$  in  $.  Then,  from  (3.4.1), 

3)  +  6y  =  b  +  (X  +  6X)  (x  +  fix)1  (x  +  6x)  +  ■£(*■  +  6x)*  b 

(2.4.2) 

Subtracting  (?.4.l)  from  (>.4.2)  and  rearranging  the  result,  we 
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where  q2  denotes  term*  of  fcecond  order  in  6 X. -.arid  6x. 

But  if  we  set  X2  =  X  +  6X,  Xj.  =  X,  x2  .=  x  +  6x  and  xi  »  x  in 
equation  (3-4-8),  we  see  that 

6Xx'x;  +  -^6x'b  '=  Qa*  .  (3-4.4) 

where  Qa'  denotes  (other)  terms  of  second  order. in  6X  and  6x. 
Thus  (3-4.3)  and- (3-4.4)  imply  that 

'  ”  .6$  -  2  x  xA6x  +  Q’i  ■:  .  (3-4.5) 

ie  6y  =  2  X  R6R  .+;  •  (3-4.6) 

since  x'x  =  R2.  Dividing  by  6R  and  letting  all  increments 
tend  to  Kero  gives 


which  is  zero  Vhen  R  =  0  and  when  X  -  0.  When:R  =  0  we  are  at . 
the  origin  and  the  value  of  y  when  R  =  0  is  the  Starting  value 
for  the  locus  of  absolute  maximum  and  absolute  minimum  y.  tfhen 
R  0,  y  is  Stationary  with  respect  to  R  only  when  X  =  0.  but 
if  X  =  0,  eiquations  (2-4)  yield,  as  Solution,  the  center  of  the 
second  order  surface,  since  we  shall  obtain  the  point  at  which 

=  0.  (i  =  l,2,...,k)  (3-3-8) 

i 

The  stated  result  follows.  Any  locus  passing  through  the  center 
of  the  surface  satisfies  (c)  or  (d) .  0therwise.it  satisfies 

(a)  or  (b). 
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4.  Comments 

Hoerl  (1959)  states  similar  but  not  quite  identical  properties 
and  ascribes  their  proof  to  Dr.  R.  Jackson  of  the  University  of 
Delaware.  No  proof ,  nor  any  reference,  is  given  in  the  paper 
however. 

The  four  results  have  the  following  implication.  If  we  wish 
to  follow  a  locus  of  the  absolute  maximum  y  for  ^Increasing  R,  we 
should  substitute  in  equation  (2.4)  only  values  of  X  greater  than 
all  the  latent  roots  of  IJ.  This  will  make  M(x)  negative  definite 
and  will  ensure  that  y  is  a  local  maximum  for  every  solution  x. 

(It  is  in  fact  an  absolute  maximum  as  we  shall  soon  see.)  No 
value  of  X  less  than  the  greatest  latent  root  should  be  consider* 
ed  in  such  a  case  for,  while  values  of  X  between  eigen  values  may 
provide  a  local  maximum  or  minimum  they  cannot  provide  an  absolute 
maximum  or  minimum. 

In  fact  the  total  range  of  X,  namely  -00  to  00  is  divided 
into  sections  by  the  latent  roots  m-x,  1x2,  ...,  Suppose 

M-i  <  M-a  <  •••  <  Then  we  have  (k+l)  intervals  (-  oo,|ix), 

(M'l^M'a)#  •••  M^)  (^»  ®)» 

As  X-*  p^(  i=l,2, .. .,k),  the  resulting  solution  x -^  +  00  so 

that  R  — co .  As  X  -*  +  00,  x  -<■  0  and  so  R  -*•  0.  Furthermore  the 
d2R 

value  of  is  positive  for  all  R  ±  O.and  is  zero  when  R  «  0. 


For  we  know  that 


* 


14 


(B  -  X  I)  x  =  -Jb 
x'x  =  R2 

Differentiating  once  with  respect  to  X  given 


.  '  "•  ...  "  ^2X» 

If  we  premultiply  (4.3-. )  and  (4'.f)  by  —  . 

ctiT- 

and  —  .  respectively,  subtract,  and  transpose,  we  find 

ST 

This,  substituted  in  (4.6),  leads  to 


_  dgR  '  _  dx'  dx 

R  5^  "  3  '-ft  -X 


(f) 


2 


Now 


dx  dx  J_- 

Thus,  using  (4.9)  in  (4.8), 

«*" 


—  -  -4-  I  x'  x  !  *  ■  .  X'  ■  d-  /(x'  x) 

ax  ^x  't5  5J  S  57  ./  lx  x; 


R>  .  3R=  ^  Si.  + 


Sx  / 

dx1  dx 


(4.1) 

(4.2) 


(B  -  X  j 

l)  ■ 

-  3x-£> 

(4-3) 

and 

x'  aS 

‘  '  dR 

_  3x 

"  Rdx  .  . 

(4.4) 

A  second  differentiation 

with  respect  to  X  gives  . 

..  ’  ■  (B  -  X  j 

r\  •  32x  p  dx  ... 

-  W  "  ' 2  $r 

■  (4.5) 

and 

'  ,  d2x  .  dx'  dx- 

-  35T  ‘ST-Sk  ~ 

Rlx^  *Hi) 

(4.6) 

(4.7) 


(4.8) 

(4.9) 


dxr  dx  ,  J  |  dx*  dx  ,  »  dx  N  2  ( 

$  $  +  -s:  ' <s  S>J  <»•» 
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The  first  part  of  the  right  member  of  (4.10)  is  always  non- 
negative  and  is  zero  only  when  R=0  or  when  ^  ■  0.  The  second 
part  of  the  right  member  of  (4.10)  is  always  non-negative  by 
a  well-known  inequality  (Hardy,  Littlewood  and  Polya,  1952)  and 
is  zero  only  when  x  *=  0,  ie  R  «  0,  or  when  ^  *  0.  When  ^  *=  0, 

x  =  0  by  (4.3)  if  X  ^  p.£,  and  thus  R  «  0.  Thus  is  positive 

except  when  R  «  0,  when  it  takes  the  value  zero. 

Note  that  «  0  does  not  imply  that  ■=  0  (and  so  that 
x  «  0  and  R  «=  0)  because  the  left  member  of  (4.4)  can  be  zero 
due  to  the  cancellation  of  positive  and  negative  cross-products. 

From  the  above,  we  see  that  the  graph  of  R,  plotted  as  ord¬ 
inate  against  X  as  abscissa,  acts  as  follows. 

AtXs-co,  R  =  0  and  R  increases  steadily  to  infinity  at 
X  =  M-iJ  between  pairs  of  latent  roots,  R  passes  down  from  infin¬ 
ity  at  through  a  stationary  value  and  up  to  infinity 
again  at'n^.  Finally  R  passes  from  infinity  at  bo  zero  at 
X  =  oo.  (See  Figure  1). 

Suppose  we  consider  what  happens  for  various  values  of  R. 

Each  value  of  R  can  give  rise  to,  at  most,  2k  corresponding  values 
of  X.  The  number  will  be  less  if  some  of  the  loops  in  Figure  1 
have  their  lowest  point  above  the  value  of  R  being  considered. 

It  is  clear  too,  that  if  we  wish  to  find  the  locus  of  the  absolute 
minimum  of  9  as  R  varies  we  can  substitute  any  values  of  X  less 
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than  the  smallest  latent  root  p-i  into  (2.4)  and  obtain  a  point 
on  the  locus ,  since  there  is  only  one  such  locus  and  thus  there 
can  be  no  ambiguity.  A  similar  remark  is  true  for  the  locus  of 
the  absolute  maximum  y  as  R  varies.  Wien  we  choose  valuer  of  X 
between  latent  roots,  however,  we  may  be  on  either  of  two  loci 
of  stationary  values  ,  depending  on  whether  we  are  to  the  right 
or  left  of  the  value  of  X  for  which  R  is  stationary. 

Ac  indicated  above,  not  all  of  the  loci  appear  for  every 
value  of  R,  but  a,.  R  increases,  more  and  vaore  appear.  Since  the 
fitted  model  can  be  considered  accurate  only  within  the  region  of 
the  experimental  design,  loci  which  do  not  appear  except  for  large 
R  are  usually  of  little  interest. 

To  summarise  the  main  practical  feature  of  this  work:  Suppose 
we  wish  to  follow  the  absolute  maximum,  predicted  value  of  y  on  a 
sphere  of  radius  R,  as  R  increases.  Find  the  latent  roots  of  B, 
choose  values  of  X  greater  then  all  of  these  roots  and  substitute 
them  into  (2.4).  Solve  for  x,  evaluate  R2  =  x*x  and  $  and  plot 
y,  xi,x2, . «,xk  against  R.  (Similar  work,  choosing  values  of  X 
lea-  then  all  of  the  latent  roots  of  13,  can  be  carried  out  for 
an  investigation  of  the  absolute  minimum  value  of  y  on  spheres  of 
radius  R) . 
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5 .  Example : 

This  example  was  used  by  Hoerl.  Consider  the  response  sur¬ 
face  in  two  factors 

y  -  80  +  O.lxx  +  0.2x2  +  0.2xi2  +  0.1x2z  +  xxx2  (5-1) 
Thus  _ 


B  = 


0.2  0.5 

0.5  0.1 


b  = 


0.1 

0.2 


(5-2) 


Equations  (2.4)  become 


(0.2  -  X)xi  +  0. 5x2  =  -  0.05, 

0.5xx  +  (0.1-X)x2  =  -0.10, 


(5-3) 


with  solution 

xx  =  (9  +  10X)  /  2  D 
x2  =  (1  +  20X)  /  2  D 


(5-4) 


where 

D  -  100  det (B  -  *  I)  -  100X*  -  30X  -  23  (5-5) 

The  eigen  values  or  latent  rootu,  of  IJ  are  given  by  D  =  0,  whence 

X  =  O.652  or  -  0.352.  (5-6) 

(Note:  Hoerl' a  parameter,  which  we  shall  call  a,  is  such  that 
X  =  b22  +  ^a,  ie  a  =  2(X  -  0.1)  for  the  example.  This  will 
lead  to  his  corresponding  eigen  values  of  a  =  1.105  and  -  0.905* 
apart  from  rounding  error.  Note  that  when  X  =  0.2,  a  =  0.2. 

In  general  putting  X  and  a  equal  to  the  same  number  would  produce 
different  stationary  points  in  the  two  calculations  and  the  fact 
that  our  calculation  below  with  X  =  0.2  produces  the  same  station¬ 
ary  point  as  Hoerl  should  have  obtained  with  a  *  0.2  is  pure 
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coincidence  due  to  the  number..;  involved.) 

If  now  we  wish  to  look  for  the  locus  of  the  absolute  minimum 
(or  maximum)  of  y  on  circles  Xj.8  +  x22  =  R2  of  radius  R,  we  should 
insert  in  equations  (5-4)  values  of  X  less (or  greater)  than  both 
eigen  values  (5-6),  ie  X  <  -  0.352  (or  X  >  0.652). 

Suppose  we  select  a  value  X  =  0.2.  Then  equations  (5-4) 
and  (5-5)  yield  solution  (xx,x2)  =  (-0.22,  -0.10);  there  is  a 
calculation  error  here  in  Hoerl's  paper.  Then  R  *  0.242,  so  that 
on  the  circle  Xj.8  +  x22  =  0.242,  y  is  stationary  at  the  point 

(-0.22,  -0.10)  but,  since  -O.352  <  X  =  0.2  <  O.652,  this  station¬ 

ary  value  y  =  79-99  is  neither  an  absolute  maximum  or  minimum. 

Continued  substitution  of  values  of  X  into  equations  (5-4) 
and  (5«5)  will  yield  four  loci  of  stationary  values  as  R  increases 
and  these,  as  evaluated  by  Hoerl,  are  shown  in  Figure  2. 

The  loci  of  absolute  maximum  and  absolute  minimum,  curves 
1  and  4,  begin  at  R  =  0  and  correspond  to  values  of  X  beginning 

at  X  =  00  and  X  =  -  cd  ,  respectively.  The  two  loci  of  intermed¬ 


iate  stationary  values  do  not  begin  until  R  =  0.195  and  correspond 
to  X  =  -0.003,  when  ^  =  0,  ie  we  are  at  the  bottom  of  the  loop 
of  R,  plotted  against  X,  which  lie*,  between  the  latent  roots 


M-i  =  -0.?52  and  p,2  =  0,652.  Because  of  the  scale  of  the  diagram, 


the  difference  in  starting  points  cannot  be  distinguished. 


The  response  surface  given  by  equation  (4.1)  is  in  fact  a 
saddle,  rising  in  the  first  and  third  quadrants  of  the  (xj,,x2) 
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plane,  falling  in  the  second  and  fourth  quadrants,  with  ridges 
oriented  approximately  45°  to  the  axes  and  with  center  slightly 
off  the  origin  at  (-  9/46,  -  1/46).  Thus  the  locus  of  absolute 
maxima  in  Figure  2  passes  from  the  origin  out  the  first  quadrant 
of  the  (xi,xs)  plane,  the  locus  of  absolute  minima  passes  out  the 
fourth  quadrant  and  the  other  two  loci  of  stationary  points, 
which  are  loci  of  neither  absolute  maxima  or  absolute  minima, 
pass  out  the  second  and  third  quadrants. 


* 


U„-. 


Figure  2  LOCI  OF  STATIONARY  VALUES  AS  R  VARIES 
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